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Abstract 



For a class of integral operators with kernels metric functions on 
manifold we find some necessary and sufficient conditions to have fi- 
nite rank. The problem we pose has a stochastic nature and boils 
down to the following alternative question. For a random sample of 
discrete points, what will be the probability the symmetric matrix of 
pairwise distances to have full rank? When the metric is an analytic 
function, the question finds full and satisfactory answer. As an im- 
portant application, we consider a class of tensor systems of equations 
formulating the problem of recovering a manifold distribution from its 
covariance field and solve this problem for representing manifolds such 
as Euclidean space and unit sphere. 



1 Problem formulation and motivation 

We start with the classical Fredholm integral equation of the first kind 



where U and V are open sets in and f : U ^ M, (7 : ^ — > M, and ijj : 
U X V ^ M. are some functions. Depending on the domains more conditions 
on /, g and ip may be necessary for the correct formulation of ([1]). Let 
{xj}*L]^ and {?/j}*L]^ be two discrete samples of points chosen by uniform 
distributions on U and V respectively. Then equation ([T]) can be discretized 
by the matrix-quadrature method 
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In general, the inverse problem of solving ([T]) for /, is often ill-posed and 
approximation based on ([2]) will eventually result in increasingly unstable 
solution as k increases. Here, however, we are interested in the first potential 
obstacle to solve ([2]) - the matrix := {ipi^iy yj)}i j=i,i ^ot be of full 
rank. The problem is stochastic one for the points Xj's and y/s are chosen in 
random fashion. In section 2 we investigated it and find some conditions for 
the kernel ip, which guarantee that for any k, has full rank with probability 
one. We also show some necessary and sufficient conditions for analytic 
kernels ip to be of finite rank. 

A further generalization of equation ([1]) takes / to be a function on n- 
manifold M and g and ip to be linear operator fields on M. For a point p & M 
with Mp we denote the tangent space at p and with Tl{Mp), the vector space 
of (l,l)-tensors (linear operators) on Mp. Let /i be a measure on M as for 
example the volume measure V{p) on Riemannian manifolds. Consider the 
equation 

/ Y{p,q)f{q)dM = C{p), pEM, (3) 

such that Y{p, .) G T^{Mp) and C{p) G T^{Mp). The inverse problem here is 
finding / for given fields Y and C. If we know that Q has a unique solution 
for / then it can be found by solving 

/ triYip, q))f{q)dM = tr(C(p)), (4) 

an equation of type ([T]). 

The importance of the class of tensor equations is that it contains 
the problem of recovering a distribution from its covariance field. Next, we 
briefiy pose this problem, while more details one can find in [1]. 

Let M be a Riemannian manifold with metric tensor G. For any p G M, 
G{p) G T'^{Mp) is a co- variant 2-tensor. Let Expp : Mp M he the 
exponential map at p and U{p) C M be the maximal normal neighbor- 
hood of p, where Exp'^ is well defined. Note that since Exp'^q G Mp, 
{Expp^q){Expp^q)'^ G T2{Mp), a contra- variant 2-tensor. For a density func- 
tion / > on M, the covariance operator field of / is GS : M — > T^M), 
such that for any p E M 

GE(p) := / G{p)iExp;'q)iExp;'qffip)dV{p). (5) 
Ju{p) 
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The problem of distribution recovering is of type if we take fi = V, 
C = GT. and Y{p,q) = G{p){Expp^q){Expp^qf . Note that 

tr{Y{p,q)) = tr{G{p){Exp~^q){Expp^q)'^) = (f{p,q), 
is the square geodesic distance on M. Thus equation (jlj), specifically, is 

I d\p,q)f{q)dV{q)=g{p). (6) 

JU{p) 

In the context of problem ([6]) we are interested in finding the rank of an 
integral operator of the form L^, : f f 'ip{p,q)f{q)dV{q), where 4'{p,q) = 
d'^{p,q) is a square distance function on M. In particular, in section 3 we 
study the rank of the Euclidean metric d{p, q) = \ \p — q\ \ and the rank of the 
standard metric on the unit sphere d{p,q) = cos~^(< p,q >), and show 
that while the Euclidean metric is of finite rank, the metric on the sphere is 
not. The last fact can be re-phrased as follows. For any discrete sample of 
points on a sphere, the square matrix of pairwise distances is non-singular 
with probability one. 

The problem of establishing the non-singularity of the kernel of operator 

is important in the context of more general statistical inverse problems 
on manifolds, as considered in [1], [11], [12] and [H]. Let g{p) = f{p) + e, 
where e is a mean zero random variable with small variance, be a model with 
unknown regression function /. For a kernel ip, the inverse problem with 
random noise is formulated as estimation of L^f from observations {pi,gi). 
In [1], Cavalier and Tsybakov estimate / from the model g = L^f + e, which 
is a noised version of ([1]). Usually points pi are assumed uniformly distributed 
on M. If the kernel ip has a finite rank, then these problems are necessarily 
ill-posed, since the operator is not invertible. 

Statistical inverse estimation is addressed in details in [12], [H], [1], [15] 
and [11]. In addition, in [2] different alternatives for regularization of ill- 
posed linear problems are discussed. In our final remarks we also briefly 
elaborate on this important aspect of all inverse problems. However, instead 
of regularizing a fixed kernel, we alleviate the ill-conditioning by choosing a 
kernel with better conditional number from a class of similar kernels. This is 
a completely different kind of approach. At the end, we show some simulation 
results supporting our proposal. 

It is a common assumption for to be Hermitian and even compact 
operator, although Mair and Ruymgaart, [TB], and Cavalier, [3], relax the 
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assumptions to Hermitian and positive definiteness. There are symmetric 
operators however, as the covariance one shown above, for which even these 
assumptions are too strong and this makes relevant the subject of this study 
- the rank of a general operator with analytic kernel. 

Remark 1 Provided that there is a local chart {U,(j)), U C M" on M such 
that (piU) contains the support of f , Y and C , equations ^ and ^ 
can be formulated as integral equations in Euclidean space by substituting 
p and q with (j){x) and (j){y) respectively. In fact, the above requirement is 
not a strong one for complete Riemannian manifolds where V{M\U{p)) = 0. 
That is why, for the sake of simplicity, to the end of this exposition we will 
work in Euclidean settings. 

2 Rank of bi-variate analytic functions 

Our goal here is to find some necessary and sufficient conditions for an in- 
tegral kernel to be of finite rank. We restrict to real analytical kernels, i.e. 
bi-variate real analytical functions. The exposition uses elementary func- 
tional analysis techniques with some textbook facts presented for the sake of 
consistency. 

Let U be an open subset of M". A collection of functions fs '■ U — > M 
is said to be hnear independent in U if '^s'^sfs{x) = 0, for almost all (by 
Lebesgue measure) x eU only if = for all s. 

Lemma 1 If functions fi{x), ...,fk{x) are linearly independent in U, then 
there exist xi, such that rank{{fi{xj)}^^ij^^) = k. 

Proof. Since fi{x) is not identically zero, there is xi G f/ such that 
fi{xi) 7^ 0. Determinant 

det ( ^'^^^^^ f^^^^^ ) = /i(xi)/2(x) - 

can not vanish for all x G f/ or fi and /2 would be linear dependent. There- 
fore, there is X2 such that det{{fi{xj)}'f^i j^i) ^ 0. The selection process can 
be extended to find xi,...,Xk in U, such that det{{fi{xj)}'^^i j^i) ^ 0. At the 
last step, defining Ai := det{{fi{xj)}^~l'j~l) ^ 0, we choose Xk E U such 
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that for X = Xk 

/ flixi) /2(xi) ... /fc(xi) 

det 

fl{Xk-l) f2{Xk~l) ■■■ fk{Xk-l 

\ fl{x) f2{x) ... fk{x) 

□ 

Definition 1 We say that function : U x V ^ M., U,V C has a rank 
k and write rank{%lj) = k if for any m E N, Xi E U and yi G V, i,j=l,...,m, 

rank{{ip{xi,yi)}'^'^j^^) < /c, 

and k is the smallest number with this property. 

Let tp : U X V ^ M., he a. smooth bi-variate function, a fact we denote 
with ip G C°°{U X V). We say that xp is analytic in V about a point p E V 
if ip has Tailor expansion in y E V 

oo 

i^{x,y) = Yl Yl c,,,„s„ix){yi-piy\..{yn-Pny", 
which for the sake of brevity we will write as 

oo 

i^{x,y) = Cs{x){y-py, 

where s = (si, Sn) is a multi-index and [s] = si + ... + Sn- 

We say that the space spanned by the functions Cg have a finite basis if 

there exist a number m and functions /i, fm, such that c^'s are linear 

combinations of /;'s, i.e. Cg E span{fi, fm} for all s. 

Next result gives some necessary and sufficient conditions for tp to have 

a finite rank. 

Proposition 1 For a function ip E C°°{U x V) that is analytic in V about 
a point p E V the following three conditions are equivalent 

(1) rank{ip) = k. 



1=1 
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(2) ip{x, y) = Yl^=i fj{^)9j{y) f^'^ linearly independent functions fj G C°°{U) 
andg^eC^{y),3=l,...,k. 

(3) There are linearly independent functions fj G C°°{U),j=l,...,k, such 
that all Taylor functions Cg are their linear combinations in U, i.e. 
Cs G span{fi,...,fk}- 

Proof. Without loss of generality we assume that p is the origin 0. We 
show that (3) (2) ^ (1) (3). 

Let condition (3) hold. Then for any s, Cs{x) = J2i=i Plfii^) ipi^x, y) = 
Ylt=iCl2s f^lu^) fii^) ^ provided that for any i and y, J^sf^sU^ converges. Sup- 
pose the contrary, that there exists io and yo E V such that Pl^y^ does not 
converge. Choose Xj G ?7,j=l,...,k, such that rank{A := {fiixj)}^'^^ ^^^) = k. 
Let > denote the norm of matrix A~^. Fix a number e > 0. Since 

all J2s ^s{xj)yQ converge, there is N such that 

M k 

I J2{J2P'Mx,M\<e/\\A~% 

[s]=N i=l 

for any j and M > N. Also by the assumption, there is M > A^, such that 

M 
[s]=N 

Define k-vectors z = {zi, ...,Zk)'^ and w = {wi, ...,Wk)'^, where 

M AI k 

■= ^^^0 and Wj := ^ (J] /^^/^(^J))?/o• 
[s]=Ar [s]=N i=l 

The system Az = w can be solved for z, z = A~^w. Then 1 12;| | < | | | l^^l | < 
e, which contradicts ||2;|| > {zi^l > e. Therefore, the initial assumption is false 
and 

s 

are well defined functions in U. Thus condition (2) holds. 

That (1) follows from (2) is obvious. Indeed, for Xi E U,yi E V, i=l,...,k, 
define k-vectors vj = {gj{yi), gj{yk))^ and = {ip{xi,yi), ...,ip{xi,yk))^ . 
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Then for the column- vectors of matrix = ip{xi,yj)}^'j^-^ we have ipi = 

Yl'j=i fji^i)'^j which shows ranki^^) = k. 

Next we show (1) =^ (3). Let ranki^l)) = k. For fixed Xi G U, i=l,...,k+l, 
consider the set 

ipixi.y), ...,ip{xk+i,y) 
of functions of y. Since no full rank matrix {ip{xi,yj)}^l^lj^i exists, by 
lemma [H {4'{xi,y)}'^^^ are not linear independent. Thus there are functions 
ai{x),x = {xi, ...,Xk+i), not all zero, such that 

fe+i 

^ai{x)'ip{xi,y) = 0,Vy e V. 

i=l 

By expanding in y we obtain 

fc+i 

^^ai(x)c,(xi)?/" = 0,V?/ G V, 

s i=l 

equivalent to 

fc+i 

^ai{x)cs{xi) = 0, Vs, 
1=1 

and this is true for any Xi G U, i=l,...,k+l. Again by lemma [H no set 
of /c + 1 functions Cg is linear independent. Therefore, there exists a set 
of functions such that for any s, Cs{x) = Yli=i Psfii.^)^ 

Cs G span{fi, fk}. Also, no less than k such functions exists, because 
otherwise, following (3) => (2) (1), we would have that rank{il)) < k. 
Thus condition (3) holds. □ 

Note that if ip is symmetric and representation (2) holds, then gi are linear 
combinations of /j. Indeed, if we choose Xj,i=l,...,k, such that rank{{fi{xj)}^^ij^^) = 
k, then we can solve the system 

k k 

^fi{xj)9i{y) = ^fi{y)gi{xj),j = l,...,k 

i=l i=l 

to obtain gi{y) = EJ=i 7ii/j(y)- 

An immediate corollary of Theorem [1] is that for tp as assumed there, 
which has a finite rank, rank{ilj) = k, the integral operator 

L^--f^ [ Hx,y)fiy)dy, f:U^R, 
Jv 
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is of finite rank. 

Example 1 Consider the Euclidean metric in M", y) = 'YTs=i{^^ ~y'^Y ^ 
where are the components of x E M". Because of the global representation 

n n 

^(x, y) = Y^ix^' * 1 - 2xV. -2xV +l * 

s=l ' I ' .=1 

by TheoremUi rank{ip) = n + 2, i.e. the Euclidean metric has a finite rank 
of 2 more than the number of dimensions. 

Definition 2 A collection of functions : f/ — > M zs said to be locally linear 
independent in U if for any open subset VofU we have that o.sfs{x) = 0, 
for almost all x eV only if as = ^ for all s. 

Note that local linear independence is stronger that linear independence and 
the former implies the latter. 

Lemma 2 If functions fi{x), ...,fk{x) are locally linearly independent in U 
and Ui C U ,i=l,...,k are open subsets, then there exist xi,...,Xk, such that 
Xi e Ui and 

rank({fi{xj)}i:^^j^^) = k. 

Proof. We can repeat the lines of the proof of lemma [1] with the only 
change at each selection step to choose Xi E Ui. 

Since fi{x) can not be identically zero in Ui, there is xi G Ui such 
that /i(xi) 7^ 0. Assuming that there are Xi G Ui, i=l,..,k-l, such that 
det{Ai := {fi{xj)}'^~^'j~i) ^ 0, we can choose Xk G Uk such that for x = Xk 

( fl{Xi) /2(Xi) ... /fc(Xi) \ 

det 

fl{Xk~l) fliXk-l) ••• fk{Xk~l) 

\ fl{x) f2{x) ... fkix) J 

Otherwise Yl'i=i ^ifii^) = 0, Vx G Uk and fi would not be locally linear 
independent. □ 

Example 2 For any k and distinct indices si, Sk, power functions {x'^'^}f^i 
are locally linear independent in any open U C M". Indeed, it is sufficient 



k~l 

= Y^Aif,ix) y^O. 

i=l 
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to show the claim for n = 1 and U = (a — 5, a + 5), 5 > 0. Assume that 
Si < S2 < ... < Sfc. Let Yl\=i'^i{^ ^ '^y^ — 0' ^ (~"^7^)- Then the 
{siY^ , {s2y^ ,---,{sk)^^ derivatives of the left-hand-side sum at x = have to 
be zeroes. Thus for any m = 1, k, 

k , 

l=m 

Which leads to the only possible choice ai = 0, for all I. 
In fact, we have a stronger result. 

Lemma 3 The power functions {a;*}[^>o ^^'"e locally linear independent in 

Proof Let U C be an open set. Consider the Hilbert space L2{U) of 
square-integrable functions in U . We have that 

L2{U) C span{x'\u, [s] > 0}, 

the closure of the span of the power functions (see [16], sec. 46). By the 
Gram-Schmidt method we can generate an orthonormal basis {0s(a;)}[s]>o of 
L2{U) from {a^^|f/}[s]>o. Since {(j)s{x)}[s\>o can not be linear dependent, so 
can not be {x*|;7}[s]>o. □ 

For subsets t/;,l=l,...,m, of M" with Ui we denote the product Ui x 

... X Um- 

Definition 3 Function ip : U x V ^ is said to have full rank almost 
everywhere (a.e.) in U x V if for any number A; G N and open sets Ui G U 
and Vj C V , i,j = 1, k we have that 

ran/c({^(xi,|/j)}^4\^.^J < k, 
\fxi G Ui and Vy^ G Vj, with an equality for at least one choice of Xi's and 

Proposition 2 Let ip G C°^{U x V) be an analytic function in V about a 
point p E V and for any /c G N, there is a set of k Taylor functions Cs{x), 
which are locally linear independent in U . Then ip has full rank a.e. inU xV . 
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Proof. Suppose that there exist k and open sets Ui C U and Vj C V, 
i,j = 1, k, such that for all Xi G Ui and i/j G Vj, rank{{ip{xi, yj)}ij=i) < k. 

If we fix Xi E Ui, i = l,...,k, then by lemma [21 {ilj{xi,y)}f^^ are not 
locally linear independent functions of y in V. Consequently there exists an 
open set W{x) C V, x = {xi,...,Xk), where {4'{xi,y)\w{x)}i=i are linearly 
dependent. Thus there are ai{x), not all zero, such that 

k 

= 0,V2/ G W{x). 

i=l 

If we expand ip{xi, ?/)'s into their Taylor series (assuming p is the origin), we 
obtain 

k 

J^ai(x)c,(xi)?/" = 0,yye W{x). 

s i=l 

By local linear independence of the power functions it follows that 

k 

^ai{x)cs{xi) = 0, Vs. 

i=l 

Since the last condition is true for any Xj G Ui, i=l,...,k, no k functions 
Cs would be locally linear indepent in U, which contradicts the theorem 
assumption. Therefore, ijj has full rank in any open subset of (S)ti U <S)ti V- 
U 

The following fact will be useful. 

Lemma 4 Any collection O of (nonempty) open and disjoin subsets ofM."' 
is countable. 

Proof. Let O G (9. Then there is at least one vector q & O with all 
rational coordinates. By applying the axiom of choice we define a map 
O H- > {ki,li,k2,l2, ■■■,kn,ln), whcrc ki, U, i = l,...,n, are whole numbers 
such that q = {2^^li, 2'^"/„) G O. O is countable since this map is injective 
one from C to Q^". □ 

Let be the Lebesque measure in M". The next measure-theoretic result 
has important consequences. 

Proposition 3 /// G C°°{U) for an open subset U ofW^, such that the set 
Vq := {x G U\f{x) = 0} has no interior points, then z/„(Vo) = 0. 



10 



Proof. Define Vi := {x G f^lf^U = 0}- When n > 1, |^ is a gradient 
vector and = means that all partial derivatives at point x vanish. 
Both Vq and Vi are closed in U. By the implicit function theorem, for any 
point X G Vo\Vi there is an open ball B^^ = B^ij-x) of radius > and a 
hypersurface of dimension n — 1 , the graph of a function of — 1 variables, 
such that X E Sx = Bx r\ Vq\Vi, Therefore there exists a collection {Sa}a 
that partitions Vo\Vi, Vo\Vi = UqS'q,, and such that are disjoin and with 
measure zero, Uni^Sa) = 0. Each 5*0, is obtained by merging countable number 
of hypersurfaces S^- Indeed, let for every x, be the support of the function 
which graph is Sx- We may assume that every Ux is a subset of one of the 
coordinate hyperplane a;* = 0, z = 1, n. Those of them that are subsets of 

= form a countable disjoin collection of open subsets, since any two Ux 
on = that intersect with each other can be seemlessly merged into the 
support of one function. 

Observe that if a; G S'q and ?/ G SpfoTa ^ P then Bx{rx/2)nBy{ry/2) = 0, 
for otherwise either x G By{ry) or y G Bx{rx), a contradiction. Therefore 
for the open sets Ua '■= U^g5^i?^(r^./2), we have Ua H Vo\Vi = Sa and [/„ 
are disjoin. By the virtue of Lemma H] the collection of open sets {Ua}a is 
countable and so is {Sa}a- Therefore z/„(Vb\Vi) = and ^'n(Vo) = i^niVinVo). 

We may repeat the same analysis for higher derivatives of order / > 1 and 
apply induction on 1. Define 

Vi := {x E U\FoT all s = (si, s„,), s.t. [s] = / : — — = 0}. 



dx'^ 



For example, let / = 2 and let 



VI := {xEU\^\x = 0}, := {x G U\^^\x = 0, for all j = 1, ...,n}. 

The above argument for showing z/.„(Vo) = i^nl^inVo) is applicable to all pairs 
(V]*, V^*) and consequently i^niVi) = ^niV^z ^ ^i*)' ^ — l,---,""- Next we use 
the fact that for any four measurable sets satisfying B G A, Vn{B) = z/„(A), 
D <Z C and Uni^D) = UniC) we have fn(-B fl = Un{A fl C), to conclude 
that Vnin'i^iVi) = Unin'^^-^Vl n^^i V^) or equivalents UniVi) = Vn{Vi n V2). 
By induction, VniYi-iW) = for any / > 0. Therefore given that 

^niVo) = l^ni^m=Oym), frOm 
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we obtain z/„(Vo) = i'n{^m=o^rn)- Finally, we realize that the set n'^-^^Vg 
should be empty, for otherwise, with all vanishing derivatives at one point, 
the function / would vanish in an open subset of U. Thus we conclude that 

By applying the above proposition for the function det{{ip{xi,yj)}^j^i) 
we immediately obtain the following result. 

Corollary 1 Under the conditions of Theorem\^ for any A; G N 

k k 

V2kn{Dk ■■= {{xi, ...Xk, yi, ■■■,yk) ^ (^U (^V : rank{{'ijj{xi, yj)}ij=i) < k}) = 0, 

1=1 1=1 

which explains "a.e." term in the full rank definition. 

Next we show that Theorem [2] can be applied to a large class of symmetric 
analytic functions. With x.y = ^''v'' denote the dot product in 

W. 

Corollary 2 Let ilj{x,y) = h{x.y), {x,y) ^ U x V and for W = {x.y\x G 
U,y & V} C M, G W . Let also h be analytic function in W about the origin 
and such that h^'^\0) ^ for infinitely many s. Then if) has full rank a.e. 
in U X V. 

Proof. Observe that 

Cs{x) = —ij{x,y)\y=o = x'h('\0). 

Since power functions are locally linear independent, for any k there exist k 
functions Cg that are locally linear independent in U. Thus the conditions of 
Theorem [2] are fulfilled. □ 

Example 3 Consider the distance on the n-sphere given by 

ip{x,y) = cos~^{x. y),x,y G S". 

Conditions of corollary \E are met for the function h{z) = cos~^(2;), z G 
[— 1, 1]. Indeed h is analytic about with infinitely many Taylor coefficients 
non-zero 

^ n ^ {2k)\ ^2/=+! 
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Example [3] shows that the distance on the n-sphere is of full rank almost 
everywhere, a fact of importance to some inverse problems on the sphere. 

Remark 2 The full rank property of cos~^ {x .y) , x, ?/ G follows from an- 
other standard result in linear operator theory. In the Hilbert space L2(S") of 
square integrable functions on S", the operator with kernel cos~^ {x .y) is sym- 
metric and thus we have the representation cos~^{x.y) = J2k>i ^k<Pk{x)(l)k{y), 
where Xk and (pk cli^g the eigenvalues and eigenvectors of this operator (see 
Theorems of Hilbert and Schmidt, fT^ . sec. 97). Since (pk 's form orthonor- 
mal system in L2(S"), they are necessarily linear independent. In fact, one 
can expect that they are locally linear independent, which will give us the full 
rank result. Our approach, however, is stronger in a sense. For example, ac- 
cording to corollarylE, cos{x.y), x,y E M", has full ranka.e., but one can not 
use Hilbert space argument to show it, since cos{x.y) is not square integrable 
in M" X W. 

3 On solvability of some systems of matrix 
equations 

Let U and V be open subsets of M"' and t] : U x U ^ M" be a continuous 
vector valued function. Let also V = {pi E f/}f=i be a set of points in 
U. For any discrete function f on V, f = {fi = f^Xi))^^^ G M.^, define 

m = {mAU 

k 

•= ^ fiVixj, Xi)T]{xj, Xi)'^ , j = l,...,/c. 

i=l 

We have G [Syrrin]^, where Syrrtn is the set of symmetric nxn matrices. 
Note that K{x,y) := r}{x,y)'r]{x,y)^ is always symmetric and positive semi- 
definite and so K is Mercer kernel (see [5]). 

If C = S[/°] for /° G M", then /° will be a solution of the system 

k 

Y,f.Y,, = C„j = l...k, (7) 

i=l 

where Yji = ri{xj,Xi)r]{xj,Xi)'^ G Syrrin- We say that /° can be recovered if 
system ([7]) has a unique solution. System ([7]) is in fact a discretization of the 
integral tensor equation 
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We re-arrange the components of each of the k"^ n x n-matrices Yji = 
{Yjr}?=i,m=v iJ=lv-,k , into new A; x A;-matrices B,m. = {Yjl^}-:^,^^, G M'=^^ 
l,m,=l,...,n. Then we order all matrices Bim vertically to obtain a.n'^kxk- 
matrix 

Similarly, matrices Cj can be unfolded in a large n^/c- vector 

[/^ll ^11^12 ^12 /^nn /^nniT 

— [(^i ...U^, U]^ ...Ufc ...Ofc J , 

and equation ([7]) can be written as 

yf = CJeW\ 

Occasionally we will write y{V) to emphasize that y is contingent on the 
choice of V. 

In fact, when C = can be recovered if and only if 

k 

Y,UHY,i)=tr{C,), j = l...k, (8) 

i=l 

has a unique solution (/°). Define 

tlj{x,y) := tr{r]{x,y)r]{x,y)'^) = \\r]{x,y)\\'^ 

and let = {'ip{xi,yj)}ij^i ^ — {'tf{Ci),...,tr{Ck)Y. Then (IHD can be 
written as f = c. The rank of function ip, which we studied in the previous 
section, determines the solvability of ([8]). 

Since Cj = G span{Yji, ...,Yjk), we have that 

rank{y\C) = rank{y), 

where 3^|C is matrix 3^ with vector C attached as a last column. The system 
of linear equations ([7]) has a unique solution if and only if y is of full rank, 
rank{y) = k. 

Therefore it is of importance to find under what circumstances y has full 
rank. If the points of V are chosen by a continuous distribution of M, what 
will be the probability for y to be of smaller rank? 

The properties of y are contingent on the choice of the vector function 77. 
It is natural first to consider the simplest choice ?7(x, y) = x—y, corresponding 
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to the Euclidean distance vector function. Observe that in this case ip{x, y) 



tr{{y -x){y -x) 



\y — x\\ and as we aheady have found in the previous 



section (Example [T]), rank{il)) = n + 2. Thus 

ranki^^) < n + 2. 

In the light of this fact, we may expect that the rank of y is also bounded 
for this choice of rj. 

Proposition 4 For the map r]{x, y) = x — y in MP, 

rank{y{V)) = m.m{k, (!1±1|!1±^}^ 

and therefore, ^ has not a unique solution for k > _ 

Proof. With respect to a fixed coordinate system in M", we identify the 
points Pi eV with vectors Vi = [vj, v^) G M". Then the matrices Yji have 
the following structure 



/ iv}-v])' {vl - v]){vf - v]) ... {vj - v]){v: - v^) \ 



V « - v^) (vj - vj) « - v^) {vf - v]) 



[V " — V 



(9) 

First we look at the case n = 1, because is much simpler and gives us 
intuition needed for the general case. We will show that for any eight real 
numbers a, b, c, d, ai,bi,Ci,di, the matrix 



X 



/ (a — ai)^ (6 — ai)^ (c — ai)^ {d — ai)"^ \ 

{^a-h,f {h-hf {c-hf {d~hf 

{a-c,f {h-c^f (c-ci)2 {d-c,f 

\{a-drf {h-dif {c-diY {d-d^Y ) 



(10) 



is singular with rank(K) < 3. Since any 4x4 sub-matrix of y has the 
structure of X, it will follow that rank{y) < 3. 

For any k x k matrix Z let PiZ) denote the j-th column vector of Z 
and C{Z) = span{l^{Z), ...,l''{Z)}, the linear space defined by the column 
vectors of Z. 
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Too see why rank(X.) < 3, we define 1 = (1, 1, 1, 1)'^, x = (ai, &i, ci, di)'^, 
= {af, b1, cf, df)'^, and consider following expression for the columns of X 

l\X) = aH-2ax + x^, 1^{X) = b'^l-2bx + x^ 

1^{X) = c^l - 2dx + x^ 1^{X) = dH - 2cx + x^. 

Then we observe that C{X) G X := span{l,x,jjf}, and therefore rank(X) < 
dim{X) < 3. 



3^ = 



check 


now the g 


eneral 


case. 


The n 


X k matrix y looks like this 


/ (v} 


-v})(v} - 
-vl){vl- 


v}) 
v\) 


(v} - 

V- 1 

{vl- 


vl)(v} 
vl){vl 


-vl) . 
-^2) • 


• (vl - vl){vl - vl) \ 

• (^2 -^fc)(^2 -^fc) 


(vl 


- vl){vl - 
-vlKvl- 

1J\ 2 


v\) 

v!) 

^1 / 


(vl- 
{vl - 

\^2 


vl){vl 
vl){vl 
vl)(v'^ 

^2 J \ 2 


-vl) .' 

2\ 

- vi) . 

-vl) . 


• ivl-vl){vl-vl) 

1 \ \ \ 1 2 2\ 

■ (vl - vl){vt - vi) 

■ ivl-vl){vl-vl) 


(vl 


-vl){vl- 


vl) 


(vl- 


vl){vl 


-vl) .' 


■ ivl-vl){vl-vl) 


(vl 
{vl 


-vl){v^- 


Vl) 


[vl- 
{vl- 


vl){v^i 
vl){v^ 


-^2) • 

-v^) . 


■ ivi-vi){v--v]:) 

■ {vl - vl){vli - vl) 




-vi){v]:- 




K- 


vl){vl 


-v^) . 


■ {vl-vl){vl-vl) 


{v^ 


-v^){vl- 
-v^){vl- 


vl) 
vl) 




v^){vl 
v^){vl 


-vl) . 
-vl) . 


■ {v- - vl){vl - vl) 

■ {vl^ - vl){vl - vl) 




-v^vl- 


vl) 


K - 


v^){vl 


-vl) : 


■ K-v'l){vl-vl) 


{v^ 




Vl) 
V?) 


X- 

'v^ - 


v^){v^ 


-v^) . 
-v^) . 


■ K-%")K-%") 

. {v^-vl){v^-vl) 






Vl) 


Yk - 


v^){vl 


-v^) . 


■ {vl-vl){vl-vl) ) 



Consider the s-th column of y. Its elements are formed by multipli- 
cation of two differences, which we will express formally as differences of 
sub-columns (1) — (2) and (3) — (4). Next we show the four sub-columns of 
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s-th column as rows 



1)^ : vl ...vl ... vl ...vi vl ...vi ... vl ...vi ... <, ...vl ... <, 



V— 

n 

k k k 



V V V 

n n n 

(3)^ : v\, ...vl ... <, ...t;," vl ...vl ... <, ... v\, ...vl ... <, ...v^ 



(4f : 

Aj /(^ /c Aj /(^ 

Formally, s-th column equals ((1) — (2)) ((3) — (4)) if the operations are taken 
comp onent- wise . 

For a vector x, with x "-^m we denote the vector obtained from x by 
shifting m positions in the right. For example, (0,1,0,0) (0,0,0,1). 
Let 

X, = i vl,...vlvl...vl...,vl...vl O,...,Of e M"'^ 

nk (n— l)nfc 

Xj = ■^(j-i)„fc, for j=l,...n, 

z, = {vl ...vlO^_^vl ...vlO^_^, 0^^^ G 

(n-l)k (n-l)fc (n-l)fc 

= ^1 ^(i-i)fc, for j=lvn, 

ii = (1_^,0^,0^)'^ G M'^'^ 

fc {n—l)k (n— l)nfc 

ij = li ^(i-i), for j=lvn, and 

iij = (ii ^(i-i)nfc) + (ii '-^(i_i)„fc+(j-i)fc), for j = 1, ...,n,i < j. 
Let /° G M"''^' be the vector 

{vlvl, vlvl, w^w^, vlvl, vlv^, w^w^. 
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2 1 2 2 2 2 



2„,2 



^l^^l, ^2^2. ^k^k, ViVi, WfcWfc, 

Then we can express s-th column of y as 

n n 



Note that 1^ 



1]^^ and therefore 



£(3^) Cspan{/0,l,^,(x^. + ^^.)}"=i, 



Finally we obtain 



dim{C{y)) < 1 + 



nin + 1) 



(n + l)(n + 2) 



n 

Another matrix of potential interest is obtained by arranging matrices 



Yji,i,]=l,...)L in one kn x kn matrix 



Z 



( Yxi 1^21 

^12 ^22 



Yki \ 

Yk2 



What is the rank of Z for the choice r]{x,y) = y — xl 
Proposition 5 For the map ri{x, y) = x — y in M", 

rank{Z) = min{/c, n{n + 2)}. 



Proof. We will find a representation of (s — l)n + m-th column of Z for 
s=l,...,k and m=l,...,n, as a linear combination of n{n + 2) basis column 
vectors and that will proof the claim. 

The elements of (s — l)n + m-th column of y are formed by multiplication 
of two differences, which we express formally as differences of sub-columns 
(1) — (2) and (3) — (4). We show these four sub-columns as rows 



(1)^ : vl ...v: vl ... vl (2)^ : vl v! 
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{3f : vl ...v[ vl ...v[ ... vl ...vl {Af : ...v[ 4, ...^ ... 4, ...vi 



n 



Then, (s — l)n + m-th column equals ((1) — (2)) ((3) — (4)) if the operations 
are taken component-wise. Let 



x"^ = (t,™, 0...0, v^, 0...0, w^, o...o)'^ e 



xf = x'l" ^(j-i), for j=l,...n, 

w = {vl, < vl, vl vl ...,v'^f e 



ii = (1,0...0,1,0...0,...,1,0...0)'^ G 



Ij = "-^0-1), for j=l,.--n, and 

Now we can write {s — l)n + m-th column of as 

n n 
^^s-l)n+m^2:) = r + J2 Vixf - V^W - ^ vilj. 

j=i i=i 
Observe that w = X]j'=i^j- Therefore 

£(Z)c.pan{r,l,,x7};4_i, 

and 

dim(C{Z)) < n + n + ri^ = n{n + 2). 

□ 

In conclusion, the rank of matrices y and Z depend on the rank of the 
bi-variate function ip. \i if) has a finite rank then the ranks of y and Z will 
be bounded as is the case with the Euclidean distance function tp{x,y) = 
\\x — As a consequence, a distribution function / in can not be 

recovered from its covariance field 

Recalling Corollary [T] we immediately obtain the following 
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Corollary 3 If ip{x,y) = tr{ri{x,y)ri{x,y)'^) has full rank a.e. in U and V 
is a finite sample of points drawn by a continuous distribution on U , then 
yiV) has full rank with probability one. 

According to Example[3|, the square sphere distance, ipi^x, y) = (cos~^(x.i/))^, 
is of full rank and Corollary [3] holds. Thus, in contrast to the Euclidean space, 
a function / on can always be recovered from the respected field 

4 Final discussion and simulation results 

Here we briefly discuss the problem of regularization of ill-posed integral 
equations of type (jl]). We re- formulate it using the geodesic distance d on a 
Riemannian manifold with volume form V 

[ d\p,q)f{q)dV{q)=g{p), U C M. 
Ju 

Discretizing the above equation on a set {pi\i of k points results to a system 
of linear equations Df = g, where D = {d^{Pi,Pj)}ij=i- Although for a 
full-rank metric d onM, D will be almost always non-singular, its condition 
number, cond{D) = \ \D\\ \\D~^\ \ , may increase dramatically as k increases, 
making solutions unstable. 

Regularization of ill-posed linear problems is an area of active research 
(see [TU], [T7], [3] and [1]). Along with the classical Tikhonov regularization 
several new approaches are currently investigated. A good overview of the 
topic can be found in [2]. 

Instead of trying to regularize an ill-conditioned kernel ip = d'^,we replace 
it with a well-conditioned element from a family of similar to it kernels. The 
procedure is applicable to all non-negative kernels. 

Consider the class of kernels {{d{p, q) — a)^}a>o- What would be a good 
element of this class that gives low (in average) conditional number for Dl 
Assuming U to be compact, we choose a that minimizes the expectation of 
((i(X, Y) — aY for X, F ~ Unif{U), and X and Y independent. It is a well 
known statistical fact that the optimal choice in this case is the expectation 
of d{X,Y). 

For example, on the unit sphere if we take U to be the whole mani- 
fold, then E{d{X,Y)) = 7r/2 and therefore, a = 7r/2 is the optimal choice 
for it minimizes E{d{X,Y) - Next we show some empirical evidence 
confirming the efficiency of this choice. 
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For a series of values of k ranging from 50 to 900 we draw 20, /c-samples on 
calculate their conditional numbers, and then report the mean conditional 
number for these 20 samples. The next table shows how the mean conditional 
number varies as k increases for the baseline choice a = and the alternative 
a = 71/2. 



a 


k=50 


k=100 


k=150 


k=200 


k=250 


k=300 






7r/2 


0.0000 
0.1291 


0.0002 
0.0710 


0.0226 
0.2865 


0.3745 
4.1476 


3.4871s 
0.5755 


1.1612 
0.1442 


X 10+15 
xlO+^ 




k=400 


k=500 


k=600 


k=700 


k=800 


k=900 






71/2 


0.0264 
3.606 


0.0694 
9.977 


0.0562 
0.757 


0.5102 
4.525 


1.5067 
8.257 


4.5686 
10.094 


X 10+19 
xlO+4 



The conditional numbers for a = increase exponentially with the increase 
of k. In fact, for values k > 250 the determinant of D has an infinitesimal 
magnitude of e~^^^ and the double-precision float arithmetic is not enough 
for consistent calculations. On the other hand, the conditional number for 
a = 71 /2 is rather stable and allows computations for k well above 1000. 
In conclusion, the simulation results are in good support of the choice a = 
Ed{X, F), X,r ~ Umf{U). 

The above described procedure can be used to improve the conditioning 
of a covariance operator. A way to achieve this is instead of definition ([5]), 
to use the following modified operator 

GEip) := [ G{p){Exp~\){Exp;\Y[l - ^^ff(p)dV{p), (12) 
Ju(p) diP^Q) 

assuming that all a{p) := E(f{p,X), X ~ Unif{U{p)) are well defined. 
Theoretical and experimental analysis of (fT2|) . however, is out of the scope 
of this paper. 
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